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ABSTRACT 

This paper rejects the hardware-based "levels of 
interaction" made popular in interactive video literature to describe 
human-machine interaction in favor of a new taxonomy of learner~media 
interaction based on the type of cognitive engagement experienced by 
learrers. Interaction can be described on three levels, based on the 
quality of the interaction* A reactive interaction is a response to 
presented stimuli, such as an answer to a specific question. 
Proactive interaction emphasizes learner construction, and generative 
activity. The learner goes beyond selecting or responding to existing 
structures and begins to generate unique constructions and 
elaborations beyond designer-imposed rules. Mutual interaction is 
characterized by artificial intelligence or virtual reality designs 
in which the learner and system are mutually adaptive, each capable 
of changing based on encounters with the otner. Reactive, proactive, 
and mutual interactivity can be described at five functional levels: 
confirmation, pacing, navigation, inquiry, and elaboration. The 
transactions (mechanics of how interaction is accomplished) can also 
be described in terms of their functions and levels of interactivity. 
Although several transactions can be employed at all le^'els of 
interaction, as interaction reaches for higher levels of engagement 
with learners, generative transactions are required. One of the major 
implications this taxonomy carries for instructional design relates 
to learner control. As levels of interaction are ascended by the 
instructional developer and reflected in the design of interaction, 
the amount of control abdicated to the learner changes. At the 
reactive level, the instructional developer retains almost complete 
control over the content, its presentation, sequence, and level of 
practice. While research in the area of learner control is relatively 
new, some tentative advice is available from the literature. Inherent 
in this emerging literature is the concept of learner control, an 
issue which will occupy a central position in multimedia research 
during this decase. (Contains 27 references.) (KRN) 
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Abstract 

This paper describes a new taxonomy of interaction based on the type of cognitive 
engagement experienced by learners, and rejects the hardware based "levels of 
interaction" made popular in interactive video literature. Reactive, proactive and 
mutual levels of interaction, and their associated funtions and transactions are 
discussed The paper also explores principles for designing interactive mult^'media 
instruction derived from the taxonomy and current research on learner control 



A Taxanonay of Interaction for 
Instructional Multimedia 

Richard A. Schwier 
The University of Saskatchewan 

This paper has two objectives. First, it will attempt to describe the need for and characteristics 
of a new taxonomy for interaction. Second, it will discuss several principles of learner control 
derived from the taxonomy and recent research 

Levels of Interactivity 

Commonly accepted labels for levels of interactivity are Level I, II, III, and sometimes IV, 
depending on your reference (luppa 1984; Katz and Keet, 1990; Katz; 1992; Schwartz, 1987; 
Schwier, 1987). These arc hardware-specific categories drawn from the literature on videodisc, 
i: nd they have achieved common acceptance (Figure 1). E^ch level depends on a particular 
equipment and software configuration. For example. Level U software is useless on a system 
which is only capable of playing Level I or III applications; Level I applications can be 
repurposcd into Level III applications by adding a computer to the system and writing a 
program to drive the application These categories are therefore confining, and not sufficient 
to describe the nature of interaction available in multimedia systems. According to this 
dcsignatiorx, every multimedia application would be a Level III or IV application, regardless of 
the nature of the program, merety because a computer is at the heart of the systemJn addition, 
these levels say little about the quality of interaction engaged by the learner. What is the 
relative quality of cognitive engagement experienced by a learner who presses buttons en an 
RCU (Level I) versus a learner who touches the screen (Level HI)? Many would argue that there is 
little difference, if any, in the level of thought required in the actions, yet they are categorized 
as dramatically different levels of interaction. 

However convenient this designation may have been or continues to be for interactive video, 
for multimedia environments it is more productive to characterize interaction according to 
the sophistication and quality of interactivity available to a learner in a particular program. 



Figure 1. Commonly accepted levels of interactivity for videodisc. 

Level I Least amount of interactivity. 

CAY or CLV discs can be used 

Program largely linear— not controlled by software. 

Picture stops maybe encoded on videodisc. 

Chapter and frame search enabled 

User can access parts of disc manually. 

Level II Intermediate degree of interactivity possible. 

Control program permanently recorded on disc. 
Program code must be compatible with playback 

system- 
Manual (RCU) control possible l:y overriding program 
Typically uses single keypad-enlry user input . 
Requires that program be submitted with edit master 

tape. 

Ideal for unchanging content 
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Level III System combines external computer and videodisc 

Ideal for volatile content or treatments. 
Higher degree of intcracti\'ity possible 
Varioufi hardware configurations poGsible. 
Computer can perform as navigator, partner, or 
pedagogue when combined with videodisc 

Level rV All visual/audio/computer sot-rces on single monitor. 

Sophisticated user interfaces such as touch screen. 
Theoretical repository for future Interactive 
innovations^ 



A Taxonomy of Interaction for Multimedia Instruction 

Multimedia, by its nature, offers a range of interactive possibilities which are remarkably 
similar, regardless of the system used to deliver the instruction. A computer acts as the heart of 
the system, and also provides the means for learners to communicate with the instruction 
Because most multimedia computer systems have similar devices for communicating (e^., 
keyboari mouse, touch screen, voice synthesisX the quality of interaction is more the product 
of die way instruction is designed, and less the result of the system on which it is delivered In 
order to describe a taxonomy of interaction for multimedia instruction, this paper will suggest 
three levels of interaction, examine functions played by interaction within these levels and 
enumerate several types of overt transactions available at each functional level of interaction- 

Figure 2, A taxonomy of interaction for multimedia instruction 



Levels 

Reactive 

Proactive 

Mutxial 



Functions 

Confirmation 

Facing 

Navigation 

Inquiry 

Elaboration 



Transactions 
Space Bar/Return 
Touch Target 
Move Target 
Barcode 
Keyboard 
Voice Input 
Virtual Reality 



Levels of Interaction 



Interaction can achieve at least three levels, based on the quality of interaction- Interaction 
can be characterized as reactive, proactive or mutual 

A reactive interaction is a response to presented stimuli, such as an answer to a specific 
question (Lucas, 1992; Thompson and Jorgensen, 1989). In other words, a learner reacts to 
given stimuli Such approaches emphasize coaching, tutorial or Socratic designs wherein the 
learner and computer are engaged in a responsive, albeit preordained, discussion. 

Proactive interaction emphasizes learner construction and generative activity. The learner 
becomes the protagonist The learner goes beyond selecting or responding to existing structures 
and begins to generate unique constructions and elaborations beyond designer-imposed limits. 

The highest level of interaction, mutual interaction, is characterized by artificial intelligence 
or virtual reality designa In such programs, the learner and system are mutually adaptive, 
that is, capable of changing based on encounters with the other. Sometimes, this is referred to 
as rccLursive interaction. Recursion is based on the nnathematical notion of indefinite 
repetition, and in mtiltimedia, it suggests a conversation which can continue indefinitely. This 
is a useful distinction, but it will fedl short of the capabilities of multimedia systems in the 
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future. We choose the term mutual because multimedia systems may ultimiately be capable of 
cybernetic conversation-actually learning from and adapting to conversation with a learner. 
At a less sophisticated level, mutual interaction can be used to describe the appearance or 
trappings of meaningful conversation. Mutual interactivity is still in its infancy, but we 
suspect this is an area into which interactive multimedia will expand 

The categories are not necessarily exclusive. Interactive multimedia programs may 
incorporate a combination of reactive and proactive approaches (few currently incorporate 
mutual approaches). The levels are hierarchical in that one subsumes the other. In other 
words. Mutual interactions contain proactive elements, and proactive interactions contain 
reactive elements. For example, when learners generate new questions and approaches 
(proactive) they can, in ttim, be used by the system to formulate new conversation (mutual). 
Similarly, when learners generate their own strategies (proactive) they are responding to 
existing stimuli at a sophisticated level (reactive). 

Functions of Interaction 

Within each level, interaction can serve several functions. Hannafin (1989) identified five 
functions of interaction: conflrmatior, pacing, inquiry, navigation and elaboration. 
Confirmation serves to verify whether intended learning has occurred, say, through leamer 
responses to embedded questions. Pacing relinquishes program time control to the learner; 
that is, the leamer determines how quickly instructional content is encountered. Navigation 
manages leamer access to instruction by facilitating access to some material ar d restricting 
access to other material Inquiry allows learners to ask questions or construct individual 
pathways through instruction, for example, through access to supplementary material 
Elaboration involves the leamer combining existing knowledge with new instructional 
content, creating transitions and contexts for moving from known to unknown information. 

Each ftinction is expressed differently during instruction, depending upon the level of 
interaction For example, reactive navigation is typified by menus or prescribed branching 
options presented to learners. Proactive navigation, by contrast, would permit the learner to 
initiate searches or participate in open-architecture movement throughout material Mutual 
navigation might happen when a program anticipates navigation routes of the leamer based 
on previous movement, and advises the leamer ^out the nature of choices made. In mutual 
navigation, the leamer could could follow or ignore the advice, and also advise the system 
about about the nature of navigation opportunities desired. Figure 3 gives one example of 
interaction obtained at each functional level of the taxonomy. These examples are meant to be 
illustrative, not comprehensive. 

Transactions During Interaction 

Transactions are what learners do during interaction; they are the mechanics of how 
interaction is accomplished For example, learners type, click a mouse, touch a screen or scan 
a virtual environment Learners can also engage in many productive types of covert 
transactions, mentally engaging themselves in the constmction of metaphors, questioning the 
validity of content, constructing acronyms to remember material and the like. This discussion 
will focus on overt transactions, but the reader should realize that covert transactions can be 
employed whenever overt transactions are unavailable to the leamer. Also, the use of one does 
not preclude the use of another. 

The level of interaction can be influenced by the type of interaction permitted by hardware 
configurations and instructional designs, and therefore tlie transactions. Several reactive 
events cannot be easily adapted to higher levels of interaction. For example, the range of 
possible interactions is confmed if a spacebar is the only method of interaction available to 
the learner, in this case, reaction would be the only possible overt interaction Devices such as 
touch screens and instmctional design strategies such as menus do not permit the leamer to 
ask unique questions or construct unrestricted paths through instruction, thereby working in 
a proactive or mutual orientation. For example, a leamer can use a touch screen or use a single 
keyboard entry to make menu selections or answer questions— a reactive level of interaction 
Touch screens and single keyboard entries are too restrictive, however, to be used for 
generative interactions such as on-line note taking— a proactive level of interaction 
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Conversely, however, transactional methods available for proactive or mutual interaction can 
also perform reactive ftinctlona For example a keyt)oard synthesizer can he used by a learner 
to compose a new song as input into a program (proactive), while the same keyboard 
synthesizer can be used to have learners play a score displayed by a program (reactive). In this 
way, transactions conform to the hierarchy of this taxonomy. Transactional events available 
for higher levels of interaction can be adapted to lower levels of interactioa but the 
relationship is not reciprocal 

Figure 4 lists several transactional events which can be employed at reactive, proactive and 
mutual levels of interactioa While the list of transactions Is not exhaustive, it Illustrates some 
interactive strategies employed in ItslI programs. The figures illustrate the notion that as 
interaction reaches for higher levels of engagement with learners, generative transactions are 
required- 
Figure 4. Some examples of transactionfi available to serve different functions and levels of interaction. 





Reactive 


Proactive** 


Mutual*** 


Confirmation 


Touch Target 
Drag Target 
Barcode 
Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Pacing 


Space Bar/Return 
Touch Target 
Barcode 
Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


NavigaticMi 


Touch Target 
Barcode 
Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Inquiry 


Touch target 
Barcode 
Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 


Elaboration 


# 


Keyboard 
Voice 

Virtual Reality 


Keyboard 
Voice 

Virtual Reality 



• Note: At a reactive level of interaction, elaboration would be restricted to covert responses to stimuli, such 
as, "Think abotit this image." Therefore, physical transactions are not required for eltijoratlon. 

••Note: Because the learner must generate original input to be truly proactive, only overt transactions which 
permit generation of complex Infcnrmation were identified In some cases, individuals might argue that 
modest forms of proactivity can be accomplished with other types of transactions. 

•••Note: Mutuality impHes sharing complex information between liscr and system- While systems may be 
able to adapt programs based on a series of single Interactions, truly mutual instruction requires complex 
dialogue. 
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Implications of ttie Taxonomy for Designing Multimedia 

The taxonomy carries implications for instructional desiga primarily concerning questions 
of learner control and instructional intent An instructional developer constantly weighs the 
need to be prescriptive versus the need for learners to explore, Theix is certainly no absolute set 
of principles, yet some guidelines and tentative principles are possible. This section of the 
--fiper will enunciate several principles for designing interactive events in instructional 
multimedia without a great deal of elaborations Original sources are identified, and the reader 
should refer to Schwier and Misanchuk (in press) for additional information. 

How does learner control converge with the proposed taxonomy? Learner controJ msy refer to 
a number of things. Learners may be granted or may require control oven 

• Content of instruction, 

• Context for learning. 

• Presentation method of the content 

• Provision of optional content 

• Sequence of material to be learned 

• Amount of practice. 

• Level of difficulty. 

• Level of advisement 

The taxonomy is meant to be descriptive, not prescriptive, yet each of these points of control 
represents a decision point for an instructional developer. As levels of interaction are 
ascended by the instructional developer, and reflected in the design of intcractioa the amount 
of control abdicated to the learner cbang'js. At a reactive l ivel of interactioa the instiiictional 
developer retains almost complete control over the content its presentation, sequence and 
level of practice. A proactive level cf interaction relinquishes much of the developer's control 
over instruction, as the learner determines what content to encounter, the sequence and how 
much time to devote to any particular element and whether additional content will be 
explored or ignored In proactive designs, the learner holds a high degree of control over all 
elements of instruction, and this may not always be beneficial to the learner. Curiously, at the 
highest level of interaction the system and the learner wrestle for control of instruction. Tlie 
learner engages the instruction ^and makes decisions, but as instruction proceeds, the system 
adopts the role of wise advisor (or tyrant) and attempts to structure the instruction for the 
learner, based on revealed needs. Thus, the amount of learner control is moderate, or shared at 
a mutual level of interaction. 

One problem for an instructional developer is to decide when to assert and when to relinquish 
control This decision will, in tura influence which level of interaction niay be appropriate to 
employ in the design of instruction. This is part of the art of instructional desiga and control 
options have increased with newer generations of multimedia systems. While *vcearch in the 
area of leamer control inlMI is relatively new, some tentative advice is available from the 
literature (Schwier and Misanchuk, in press). 

General Conclusions About Control 

• Leamer control may increase motivation to learn (Santiago and Okey, 1990; Steinberg, 
1977). 

• Leamer control does not necessarily increase achievement and may increase time spent 
learning (Santiago and Okey, 1990). 

• Leamer control may permit students to make poor decisions about haw much practice 
they require, which are reflected in decremented performance (Ross, 1984). 
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Ccwitrgi Issues Related to Learner Characteristics 

• Learners who are generally high achievers or who are knowledgeable about an ai'ea oi 
study can benefit from a high degree of learner control (Borsook, 1991; Gay» 1986; 
Hannafin and Colamaio, 1987). 

• Naive or uninformed learners requii"e structxire, interaction, and feedback to perform 
optimally (Borsook, 1^1; Carrier and Jonassea 1988; Higginbotham-Wheat. 1988, 
1990; Kinzie, Sullivan, and Berdel, 1990^ Schloss, Wisniewski. and Cartwright 1988). 

• The effectiveness of learner control is mitigated by such learner characteristics as 
ability, previous knowledge of the subject matter, and locus of control (Santiago and 
Okey, 1990). 

Control Issues Related to Program Variables 

• LeaiTier control with advisement seems to be superior to unstructured learner control for 
enhancing achievement and cuiriosity, promoting time-on-task^ and stimulating relf- 
chalienge (Amone and Grabowski, 1991; Hannafin, 1984; Mattoon, Klein, andlTiurman, 
1991; Milheim and Azbell, 1988: Ross, 1984; Santiago and Okey, 1990). 

• Learner control of presentations has been shown to be beneficial with respect to text 
density (Ross, Morrisoa and O'DeU, 1988) and context conditions (Ross, Morrison, and 
O'DeH 1990). 

• Courseware should be adaptive. It should be able to alter instruction dynamically, based 
on learner idiosyncracles (Borsook, 1991; Carrier and Jonassea 1988). 

• One opinion holds that learners should be given control over contextual variables such 
as text density, fonts, and backgrounds, but not over content support variables such as 
pacing, sequence, and examples (Hlgginbotham- Wheat 198ft 1990). 

These results, however inviting, should be approached with caution. Not only are they 
Inconclusive, they are contradictory in some cases. For example, the advice offered by 
Higginbotham-Wheat ( 1988; 1990) can be interpreted to mean that learners should influence 
only variables which have little insti-uctional significance, and be denied control of 
significant instructional variables. Certainly this contradicts the intentions and findings of 
many of tlie other studies cited Indeed, some argue that we need to go beyond objective and 
prescriptive designs, and embmce generative and constructivist approaches (Jonassen, 199 IX 
Inherent in these u'guments is the concept of control an Issue which will occupy a central 
position in multimedia research dioring this decade. 
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